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ABSTRACT 

How nursing science researchers conducting mixed 
method triangulation are using the heterogeneity of irrelevancy 
assumption in practice was studied. Seven questions were developed to 
reflect the implications for practice of the heterogeneity 
assumption. Answers to these questions were obtained from 23 written 
cases (14 articles, 1 presentation, and 8 dissertations) of mixed 
method triangulation in nursing science published or conducted during 
1980 through 1990. Data were analyzed using techniques from grounded 
theory. Primary analysis consisted of open coding and constant 
comparison to develop conceptual labels and more abstract conceptual 
categories related to each question. A second analysis involving 
selective coding and constant comparison is incomplete, but was 
intended to develop the hypothesis regarding the core nature of mixed 
method triangulation in nursing science. Results suggest that nursing 
science researchers are practicing a form of triangulation that does 
not conform to the directives and constraints of the heterogeneity of 
irrelevancy assumption. The concept of triangulation that they are 
practicing may be driven more by substantive than methodological 
concerns. Examples of rationales for method selection, meta-outcomes , 
and resolutions of divergent results may provide guidelines for 
practitioners of mixed method triangulation. An appendix lists the 
five mixed method purposes defined by J. C. Greene and others (1989). 
There is a list of 46 references. (SLD) 
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Abstract 

The primary purpose of this study was to investigate how 
nursing science researchers conducting mixed method triangulation 
are using the heterogeneity of irrelevancy assumption in 
practice. Seven questions were developed to reflect the 
implications for practice of the heterogeneity of irrelevancy 
assumption. Answers to the seven questions were obtained from 23 
w:ritten cases of mixed method triangulation in nursing science* 

Data were analyzed using techniques from grounded theory 
(Strauss & Corbin, 1990). The primary analysis consisted of open 
coding and constant comparison to develop conceptual labels and 
more abstract conceptual categories related to each question. A 
second analysis involving selective coding and constant 
comparison is incomplete but was intended to develop hypotheses 
regarding the core nature of mixed method triangulation in 
nursing science. 

Results suggest that nursing science researchers are 
practicing a form of triangulation that does not conform to the 
directives and constraints of the heterogeneity of irrelevancy 
assumption. Rather, researchers in nursing science may be 
practicing an alternative conception of triangulation that is 
driven more by substantive rather than methodological concerns. 
Data based examples of rationales for method selection, meta- 
outcomes and resolutions of divergent results may provide 
guidelines for practitioners of mixed method triangulation. 
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MIXED METHOD TRIANGULATION: THEORY AND PRACTICE COMPARED 
* This paper is a working draft of ideas about mixed method 
triangulation. It is one step in the long process of 
formulating, clarifying and writing ideas, I would appreciate 
any suggestions and feedback you have for refining the ideas and 
for improving their presentation. 

Introduction 

Researchers and practitioners in the social sciences voice 
concern over a gap between theory and practice. While the 
heterogeneity of irrelevancy assumption has received much 
attention in theoretical discussions of triangulation, it is 
questionable whether it is relevant to triangulation as 
practiced. The current investigation was conducted in an effort 
to help bridge the gap between theory and practice. 

Thus, the overriding goal of this investigation was to 
compare the practice of mixed method triangulation with the 
theory of mixed method triangulation based on the heterogeneity 
of irrelevancy assumption. In the process of making this 
comparison, the primary purpose was to evaluate if and how 
researchers in nursing science are using the heterogeneity of 
irrelevancy assumption in their work. A second purpose was to 
try to develop concepts and categories that were grounded in the 
particulars of data but abstract enough to be applicable to 
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disciplines other than nursing science. A third purpose was to 
develop hypotheses for future research regarding the nature of 
mixed method triangulation in nursing science. 

This investigation builds on the work of many others in the 
area of mixed method research. However, two references are of 
particular importance. Greene, Caracelli and Graham (1989) have 
identified triangulation as one of five purposes of mixed method 
research. Greene, Caracelli and Graham's five purposes of mixed 
method research are defined in Appendix A. In her essay, "Why 
Triangulate?", Hathison (1988) has challenged us to question the 
applicability of the conventional notion of triangulation based 
on the heterogeneity of irrelevancy assumption. The focus of 
this study is on the practice of mixed method triangulation as 
identified by Greene et al. (1989) in relation to the 
conventional notion of triangulation as discussed by Mathison 
(1988) . 

In literature pertaining to mixed method research, 
triangulation is commonly associated with the heterogeneity of 
irrelevancy assumption. This assumption indicates that 
irrelevant error associated with individual methods can be 
canceled out in a mixed method triangulation design if methods 
with counteracting biases are focused on the same entity of 
investigation and implemented independently. When error is 
controlled through compensation of inherent biases and different 
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methods produce the same results^ it is often assumed that what 
is left is more valid findings (Campbell & Fiske, 1959; Collins, 
1981; Jick, 1979). 

The heterogeneity of irrelevancy assumption has definite 
implications for the practice of mixed method triangulation. It 
is questionable whether these implications are applicable to 
practice and whether researchers are actualizing these 
implications in practice. The primary purpose of this 
investigation is to determine if and how nursing science 
researchers conducting mixed method triangulation seem to address 
the implications associated with the heterogeneity of irrelevancy 
assumption in practice. These implications are reflected in the 
seven research questions that guide this investigation. The 
research questions are as follows: 

1. What kinds of rationales are given for selecting sets of 
methods with counteracting bias? 

2. To what extent are the hypothetical biases of methods 
preserved by implementing different methods independently? 

3. Are researchers assuring that different methods are 
focused on the same entity of investigation by implementing 
methods simultaneously or by establishing the stability of the 
entity over time? 

4. What reasons or evidence are given by researchers for 
believing that questions asked by different methods and 
information obtained by different methods are comparable? 
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5<, How do researchers justify the expectation of 
convergence as more valid findings and determine its existence? 

6. What kinds of outcomes do actually result from attempts 
at mixed method triangulation? 

7. How is nonconvergence resolved? 

Methodology 

Mixed method research as an area of investigation is 
relatively new. Much has been learned about what questions to 
ask and what benefits to expect from the first hand experience of 
researchers who have conducted mixed method research. Researchers 
like Greene, Caracelli and Graham (1989) and Rossman and Wilson 
(1983, 1991) have compiled and analyzed their own and others' 
mixed method studies to develop conceptual frameworks for 
conceptualizing and practicing mixed method research. Research 
about mixed method research seems to be focused on conceptual 
understanding and the development of theory. Similarly, the 
emphasis in this study is on the development of concepts related 
to mixed method theory while "testing" (conceptually rather than 
s^tatistically) the applicability of the heterogeneity of 
irrelevancy assumption to mixed method practice in nursing 
science. 

Grounded theory (Strauss & Corbin, 1990) is a theory 
building research methodology. It has been selected as the 
primary methodology in this investigation for several reasons. 
First, grounded theory is based on the idea that theory should be 
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developed from the particulars of practice. This is consistent 
with my belief that theory is important to the development of a 
discipline, but that for theory to be meaningful and applicable, 
it needs to be grounded in the "reality" of practice. Second, 
the "reality" of practice in this investigation is the "written 
worlds" (articles, dissertations and unpublished papers) of 
practitioners of mixed method triangulation in nursing science. 
Grounded theory is useful here because it is so easily applied to 
documents. Third, grounded theory is devoted to the development 
of concepts, categories and systems related to a phenomenon. One 
goal of this investigation is to contribute to the development of 
theory. While the concepts developed here are based in data from 
nursing science, it is hoped that the ideas will be abstract 
enough to be applicable to mixed method research in other 
disciplines in the social sciences. 

Grounded theory is most frequently applied to a global area 
of study. "What is important" in a general sense is allowed to 
emerge from the data. However, in this investigation, grounded 
theory was not applied at such a global level. Rather, the 
"global" areas of study were focused by seven research questions. 
Grounded theory was applied to the data collected about the seven 
research questions. Thus, "what was important" on a global level 
was predetermined by the seven research questions. "What was 
important" in the context of each question was allowed to emerge 
from the data. Furthermore, grounded theory usually involves the 
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development of concepts, categories and statements of 
relationships among concepts and categories. In this 
investigation, data collection and analysis were limited to the 
development of concepts, categories and hypotheses about possible 
relationships among concepts and catagories. 

Data collection . The data base for this investigation 
consists of 23 cases of mixed method triangulation in nursing 
science. Cases were purposively selected for inclusion in the 
study if they consisted of single studies; produced at least two 
separate data sets; stated an intent to collect, compare and/or 
combine words and numbers about the same entity of investigation 
for the purpose of obtaining convergence; and were published or 
conducted during the years of 1980 through 1990. The sample 
included 14 articles, 1 presentation and 8 dissertations. Cases 
were located through word of mouth, citations in literature, hand 
searches in journals, and computer searches of six different data 
bases: Dissertation Abstracts, Nurse Search, Medline, Psychlit, 
ERIC, and Sociofile. Computer search terms included* 
triangulation, qualitative/ quantitative, mixed method(s), 
multimethod(s) , and mult ipl ism. 

In this investigation, mixed method triangulation refers to 
the process of combining qualitative and quantitative methods 
focused on the same entity of investigation for the purpose of 
obtaining convergence of results. As noted by Greene et al, 
(1989), triangulation may be of primary or secondary importance 
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in a mixed method study. Both degrees of importance are 
reflected in the cases selected for this investigation. In 39% 
of the cases triangulation seeiaed to be the primary mixed method 
purpose in that the design of the study seemed to revolve around 
the triangulation purpose. In 61% of the cases, triangulation 
seemed to be a secondary mixed method purpose. In these latter 
cases, triangulation seemed to be pursued as a strategy in 
situations where other mixed method purposes (i.e., 
complementarity, development, initiation, expansion (Greene et 
al., 1989)) were dominant. 

Procedures . An initial set of seven research questions was 
developed representing implications for practice based on the 
heterogeneity of irrelevancy assumption. The questions were 
designed to reveal if and how the conventional theoretical notion 
of triangulation was being acknowledged and utilized by 
practitioners. Information pertaining to the seven questions was 
collected from each written case of mixed method triangulation. 
The information regarding each question was then grouped together 
and analyzed as separate categories of information. The primary 
analysis involved open coding and constant comparison (Strauss & 
Corbin, 1990) to develop concepts^ categories and hypotheses for 
further research. After the answers to each question were 
analyzed per question, all information was combined and a second 
analysis was started. The second analysis involved the use of 
selective coding and constant comparison (Strauss & Corbin, 1990) 



10 



Mixed Method Triangulation 

Page 10 

in an atteiapt to reveal a basic nature of the practice of mixed 
method triangulation in nursing science, something that might 
characterize an alternative conception of triangulation. At this 
point in time, the second 'across-question' analysis is 
incomplete. Most of the results in this paper are based on the 
primary 'question-by-question' analysis of the information 
obtained from cases of mixed method triangulation in nursing 
science. 

Significance. Ideas and hypotheses that are developed in 
this study of mixed method triangulation in nursing science may 
be applicable to other disciplines in the social sciences where 
researchers are investigating attitudes and beliefs of human 
beings, evaluating the effects of intervention and programs, and 
developing instrumentation and methodology. Data-based examples 
of concepts developed in nursing science may provide other 
researchers in the social sciences with some guidance for 
practi^.ing triangulation • In addition, the results of this 
investigation based on data from nursing science may provide 
mixed method researchers in the social sciences with some reasons 
to question or accept the utility, implications and conception of 
triangulation based on the heterogeneity of irrelevancy 
assumption. 

Limitations. This study is based on the written worlds of 
nursing science researchers. Primary reliance on unobtrusive 
document analysis in this study precludes confounding of data due 
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to reactivity and the act of measurement. However, there is less 
of an opportunity to check the accuracy of written reports and 
investigate the unwritten motivations of authors. 

In this investigation, an attempt has been made to collect 
and analyze as many cases of mixed method triangulation in 
nursing science as possible, it is not assumed that all possible 
cases have been collected. Thus, the data base is subject to 
revision as are the resulting ideas and hypotheses. Furthermore, 
the generalizability of results based on nursing science data is 
limited to the extent that particulars of mixed method piactice 
are differential across disciplines. 

The intent of this investigation may best be understood in 
terms of its dolimitations. First, it is not the intent of this 
r>tudy to achieve a level of specificity indicative of logics in 
use. That level of specificity might best be obtained through 
the use of interviews. Second, it is not the intent of this 
investigation to test statistical hypotheses, but rather to 
explore and develop concepts and "working" hypotheses for further 
investigation. Third, it is not the intent of this investigation 
to generate statements of relationships among concepts, although 
relationships among concepts constitutes an excellent area for 
further research. Fourth, it is not the intent of this 
investigation to address or resolve debates regarding mixed 
method triangulation at a philosophical level. Finally, it is 
the intent of this investigation to simply produce a data-based 
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analysis of some issues relevant to the practice of mixed method 
triangulation in nursing science in the process of comparing 
theory and practice. 

Author ^s npte. One obstacle to the conceptualization and 
practice of mixed method research may be the qualitative verses 
quantitative debate in the social sciences. It is now considered 
appropriate in most educational research circles to combine 
qualitative and quantitative "techniques and procedures" (Smith & 
Heshusius, 1986). However, the issue of whether or not "research 
paradigms" can be combined fuels much of the remaining debate 
(Howe, 1990). 

In spite of this philosophical contention about the 
inf easibility of combining research paradigms, people are 
pursuing the fusion and parallel implementation (Louis, 1982) of 
diverse methods and methodologies. It is assumed by this author 
that much is to be learned about the theory and practice of mixed 
method research by studying the written reports of others' 
attempts at combining methods and methodologies. Perhaps it is 
by looking at what actually has been done in the way of mixed 
method research that issues raised in the qualitative- 
quantitative debate may be addressed in the empirical sense. 
Theory and Practice Compared; Regul tg 

In the spirit of a chart essay approach (Jones & Mitchell, 
1990), results of this study are presented for each of the seven 
research questions. Each question is stated and followed by a 
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brief discussion of the theoretical issues it represents, the 
answers to the particular question based on the information 
obtnined from the 23 cases of mixed method triangulation in 
nursing science, and a brief discussion of the results and 
corresponding implications per question. In the summary section, 
information across questions and cases is briefly summarized, 

1. What kinds of rationales are given for selecting sets of 
methods with counteracting bias? 

Issues. Convergence as "truth" or as more valid findings 
seems to be justified in theory when sets of different methods 
with counteracting biases are implemented in a way that preserves 
their error and allows for compensation of that error. Thus, 
when methods are very different in terms of their sources, 
directions and magnitudes of associated error, convergence may be 
indicative of validity or trustworthiness of constructs, findings 
or interpretations. When methods tend to be similar in regard to 
error, convergence may simply imply that the methods are 
interchangeable or reliable with regard to one another. Whether 
methods are similar or different in regard to bias, there is also 
the possibility that convergence could be the result of 
compounded bias or error. Considering that the theoretical 
meaning of the results of mixed method triangulation depends upon 
the degree to which the methods are different in regard to their 
sources, magnitudes and directions of error, it seems important 
to select a set of methods for a mixed method triangulation 
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design in a very strategic way. Of interest here is how 
researchers identify the hypothetical bias associated with 
individual methods and how researchers construct sets of methods 
that possess counteracting bias. 

Results and implications. Researchers in eight of the 
twenty-three studies (35%) stated rationales for the selection of 
their sets of quantitative and qualitative methods. In all but 
one case, rationales seemed a priori in nature. That is to say 
that the justification of the me^-hod set seemed to precede the 
implementation of the methods and the interpretation of the 
results. In one case, the rationale was developed post hoc in 
the sense that the convergent results were used to support the 
selection of methods. Three kinds of rationales for selecting 
sets of methods emerged from the data: substantive, 
methodological and logistical. 

Substantive rationales were based on an analysis of the 
substantive or informational strengths and weaknesses of 
different methods rather than on an analysis of the hypothetical 
methodological biases associated with each method. Glik, Parker 
and Muligande (1987, p. 182) provide an example of a substantive 
rationale: 

"Kroeger states that research which uses 
anthropological approaches tends to yield information 
about health decisions-making processes as well as an 
understanding, of the disease/illness experience within 
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a specific cultural context. Yet this approach may not 
link that information to overall health care 
utilization patterns. On the other hand, 
epidemiological surveys may yield information about 
disease prevalence, utilization rates, and access to 
care factors, but may not account for biases and 
assumptions implicit in Western constructs of disease 
and illness or take into account the contexts in which 
health care decisions are made." 

Six researchers created substantive rationales for selecting 
sets of methods. In all cases, the purpose of substantive 
rationales seemed to be to show how different methods would 
generate different kinds of information in order to yield a more 
comprehensive picture of the entity of investigation as a result 
of combining methods. This search for substantive diversity and 
complementarity seems inappropriate in the context of 
triangulation which seeks the sameness of information from 
different methods. 

Methodological rationales involved an analysis of the 
methodological strengths and weaknesses of different methods in 
terms of potential bias and error rather than in terms of 
substantive possibilities. Methodological rationales were stated 
in six cases. In all six cases, the purpose was to show how the 
methodological weaknesses of one method were compensated for by 
the methodological strengths of another method. Hinds and Young 
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(1986, p. 196) provide an example of a rationale that contains 
primarily methodological reasons for selecting their specific 
combination of methods: 

"The methods were viewed as compensatory, as the 
limitations of one were offset by the strengths of 
another. For example, archival records were used to 
counterbalance the reactive influence an investigator's 
presence could have^n a participant's self report 
data. Observation data, which can be contaminated by 
observer bias, were compared with or checked against a 
participant's questionnaire and interview responses. 
Field notes were used to expand and explain quantified 
questionnaire responses. Open ended interview 
questions offset the predetemnined, defined, and 
limited foci of the questionnaires. The purposeful 
selection of these data collection methods reflected 
the assumption that every method is subject to certain 
validity threats .... In essence, the selected methods 
were assessed as having dissimilar biases and therefore 
were thought to diminish the systematic effects of 
participant and investigator based errors associated 
with any single method." 
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Logistical rationales involve an analysis of the logistical 
strengths and weaknesses associated with a set of methods. Glik 
et al. (1987, p. 182) provided the only example of a logistical 
rationale for method selection: 

"... Large scale in-depth surveys or classical 
anthropological field work approaches are costly and 
time consuming, and because of this, they are often not 
feasible tools for health education planning. An 
alternative approach is to use multiple methods of data 
collection, which can maximize the validity of data to 
be collected in the field and minimize the cost, by 
allowing a smaller sampling frame for in-depth phases 
of research." 

While time and money were indicated as logistical concerns in 
this rationale, it is conceivable that political concerns may 
also be important in the selection of sets of methods. 

Summary . Thirty-five percent of the researchers cited 
rationales for method set selection. These researchers created 
three kinds of rationales: substantive, methodological and 
logistical. Examples of these three kinds of rationales may 
provide guidance to other researchers concerned with justifying 
the selection of their methods. One area for future research is 
the association of different rationales with different purposes 
and contexts for mixed method research. For instance, it may be 
that substantive rationales are more likely to be found in 
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situations where (a) the primary mixed method purpose has to do 
with substantive comprehensiveness rather than with 
methodological triangulation, (b) mixed method triangulation is 
used as a strategy rather than as an overall purpose of the mixed 
method study, and (c) mixed method triangulation is emergent 
rather than preplanned. 

Sixty-five percent of the researchers did not cite reasons 
for believing that the set of methods they used did in fact 
possess counteracting bias. Several issues seem worth 
investigating. First, it may be wrong to assume that researchers 
who did not include rationales for method selection in their 
reports did not have rationales. Restraints imposed in the 
publication process may have restricted the incorporation of 
rationales into reports. Other reasons may have precluded the 
inclusion of a description of the rationales for method 
selection. 

Second, the lack of rationales brings to question whether 
the creation of rationales for* method selection is viewed by 
researchers as an important design feature of mixed method 
triangulation. If it is not, perhaps researchers are unaware of 
the implications associated with the heterogeneity of irrelevancy 
assumption. Perhaps researchers have found it impossible to 
specify hypothetical strengths and weaknesses of different 
methods or perhaps they are disregarding the idea that bias or 
error can cancel out in a mixed method design. Or, perhaps they 
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are responding to a conception of triangulation that does not 
depend on the heterogeneity of irrelevancy assumption. 
Interviews would be very helpful for revealing i ^. •: oerception or 
conception of mixed method triangulation from the researcher's 
point of view. 

2, To what extent are the hypothetical biases of methods 
preserved by implementing different methods independently? 

Issues. The preservation of inherent biases in individual 
methods is a prerequisite condition for assu:?ing that bias in the 
overall design has been canceled out. By enacting independent 
implementation of methods, researchers may infer in theory that 
they have preserved the counteracting biases associated with the 
different methods (Campbell & Fiske, 1959; Greene et al., 1989' 
McClintock & Greene, 1985). Independent impleraentation refers to 
the process of conceptualizing, designing and implementing 
methods independently (Greene et al., 1989). 

Theoretically, independent implementation affects how one 
can interpret the results of mixed method triangulation. If 
different methods are implemented independently, the results of 
triangulation may be interpreted in light of the assumption that 
bias has been canceled out. If bias is assumed to be canceled 
out, convergence may indicate more valid findings (Campbell & 
Fiske, 1959) and nonconvergence may provoke a substantive 
reconceptualization. However, if methods are not implemented 
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independently, researchers may have a more difficult time 
assuming that bias has been canceled out, and the results of 
triangulation may therefore be more equivocal. 

Results and implications. The notion of independent 
implementation was not addressed directly in any of the cases of 
mixed method triangulation in nursing science. It was difficult 
to determine whether methods were implemented independently or 
interactively in seven cases (31%). In fifteen cases (65%), 
methods seemed to be implemented interactively. In the majority 
of cases, one researcher seemed to have designed, implemented and 
analyz'^d both qualitative and quantitative methods • 

In only one case (4%) was it obvious that methods were 
implemented independently. In a case by Fagan (1988), different 
people implemented different methods to assess the same 
phenomenon, peripheral perfusion. Perhaps independent 
implementation was feasible because there were multiple 
researchers. 

There may be several explanations for the lack of 
independence in implementation of methods. First, researchers 
may not be clear about what independent implementation means in 
operational terms. Or, independent implementation in operational 
terms may not be easy to obtain, especially in a situation where 
only one researcher is conducting a study. Second, researchers 
may be rejecting the theoretical importance of independent 
implementation or at least not recognizing it as important in 
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their work. Third, other design features may be more important 
to researchers in designing mixed method triangulation. By 
interviewing researchers, these other more important features 
might be specified. 

A side issue for future investigation is the idea that 
independent implementation was not addressed in any of the cases 
regardless of whether triangulation was planned or emergent. It 
seems logical that independent implementation might not be 
feasible in cases where triangulation is emergent because it 
would be difficult to plan ahead for all possible cases of 
triangulation in an emergent design. However, it is somewhat 
more confusing to find that independent implementation is not 
addressed in planned triangulation where it is possible to set up 
conditions ahead of time that would affect the interpretation of 
results. 

Summary . Independent implementation of methods is not 
©ddressed or enacted by most researchers conducting mixed method 
triangulation in nursing science. In one case where independent 
implementation was utilized, different researchers implemented 
the different methods. Further research is needed to determine 
why the issue of independence is not addressed. 

3. Are researchers assuring that different methods are 
focused on the same entity of investigation by implementing 
methods simultaneously or by establishing the stability of the 
entity of investigation over time? 
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Issues, A distinguishing feature of triangulation is that 
one entity is being investigated* In triangulation, different 
methods are focused on the one entity of investigation with the 
hope that the results or findings of different methods will 
agree • While there is no defacto way of knowing if methods are 
actually focused on the same entity, researchers may increase the 
theoretical likelihood that methods are focused on the same 
entity if the methods are implemented simultaneously (Greene et 
al., 1989; McClintock & Greene, 1985) or if researchers can 
establish that the entity of investigation is stable over time 
(Greene et al., 1989). In either case, it seems that researchers 
need to have some assurance that different methods are not 
focused on different aspects of a changing entity or on different 
entities. Theoretically, if methods are focused on different 
entities, convergence might represent an accumulation of error 
and nonconvergence might represent a picture of "truth" . 

Results and implications. The notion of simultaneous 
implementation or stability of the investigative entity was not 
addressed directly in any of the cases of mixed method 
triangulation in nursing science. In three cases (13%), it was 
possible to determine that methods were not implemented 
simultaneously. In nineteen cases (83%), it was difficult to 
determine whether or not methods were implemented simultaneously. 
The notion of the stability of an entity of investigation over 
time was not addressed by researchers in any of the cases. 
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In one case (4%) by Fagan (1988), the methods were obviously 
implemented simultaneously. This is the same case in which 
independent implementation of methods was noted. In this 
situation several people implemented different methods 
simultaneously and independently. When the results of the 
different methods converged on the same diagnosis, the primary 
researcher justified the reliability of her methods and the 
validity of the diagnosis. 

In spite of the lacking evidence for simultaneous 
implementation or stability of entities, several researchers did 
address the issue of whether or not methods were focused on the 
same entity. Aroian & Patsdaughter (1989) stated that scores, 
verbal self -reports, clinical assessments and cultural- 
environmental assessments were all directed at the phenomenon of 
psychological distress. O'Brien (1990) stated that a 
standardized scale and focus interviews both revealed patient 
compliance behavior. Perhaps the researchers in these two 
situations were implying that the entities of investigation were 
stable over time. If this were true, methods might not need to 
be implemented simultaneously to insure that the same entity was 
being measured by all methods. 

Hinds and Young (1987) obtained positive significant 
correlations between quantified content analyzed concepts from 
interviews and scores from self -report questionnaires. They used 
these correlations to support their theoretical contention that 
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qualitative methods and quantitative methods were "indexing the 
same construct (i.e., wellness)" (p. 197). They also used these 
correlations as evidence of convergence and validity of findings. 

The use of correlations for both purposes is interesting. 
According to theory, it seems that researchers might first want 
to establish that methods are focused on the same entity before 
comparing the results of the different methods to determine if 
convergence has occurred. To use the correlations as evidence of 
one entity of investigation and of the convergence of results is 
one possibility. To use the correlations as evidence of 
convergence in a situation where it has been established through 
some other means that one entity of investigation is being 
pursued is another possibility. 

Summary > Simultaneous implementation and stability of 
constructs were not addressed directly by nursing science 
researchers in their written cases of mixed method triangulation. 
In the one case where simultaneous implementation was utilized, 
several researchers were available to actually implement the 
different methods simultaneously. It might be that multiple 
researchers are needed to implement methods simultaneously and 
independently. Simultaneous and independent implementation of 
methods may not be practical in situations where mixed method 
triangulation is conducted by one researcher. This issue has 
been discussed by McClintock and Greene (1985). Further research 
is needed to determine the feasibility of simultaneous and 
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independent ixnpleinentation in practice, the meaning of 
simultaneous and independent implementation to researchers, and 
ways to establish that one construct is being investigated by all 
methods . 

4, What reasons or evidence are given by researchers for 
believing that questions asked by different methods and 
information obtained by different methods are comparable? 

Issues. In mixed method triangulation, qualitative and 
quantitative methods and information are combined within one 
study. Rossman and Wilson (1985) refer to qualitative methods as 
those that "... usually generate data not easily reduced to 
numbers ..." (p. 628) and quantitative methods as those "... to 
which the power of mathematical analysis can be applied easily" 
(p. 628). Similarly, Greene et al. (1989) define "... mixed 
method designs as those that include at least one quantitative 
method (designed to collect numbers) and one qualitative method 
(designed to collect words), where neither type of method is 
inherently linked to any particular inquiry paradigm" (p. 256). 
An issue for mixed method triangulation is whether the 
qualitative and quantitative methods being used are responding to 
the same questions and whether the information obtained from 
qualitative and quantitative methods can be compared. 

Results and implications. In eighteen of the cases (79%), 
researchers did not address the issue of comparability of 
questions or information. In four cases (17%), researchers 
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addressed the issue indirectly: (a) Jones, Strom and Daniels 
(1989) indicated that structured interviews had been designed to 
assess the content validity of the Parental Strengths and Needs 
Inventory in the deaf culture. The authors implied that the 
interview questions corresponded to the topics covered on the 
inventory. (b) Hildman (1987) stated that a telephone interview 
was designed to match the information requested in a mail survey. 
The interview and survey were made available to readers, but an 
analysis of the cor,r>arability of questions or data was not made 
explicit. (c) Similarly, Rovjatt (1983) stated that scales and 
interviews addressed similar questions. She, too, made the 
protocols available in the appendix without an analysis of the 
comparability of questions or information. (d) Fagan (1988) was 
a little more direct in that she offex'ed some explanation of how 
different methods were assessing different indicators of the same 
entity of investigation, shock. 

Hinds and Young (1987) were the only authors to deal 
directly with the comparability of questions and information. 
Their investigation provides a good example of a preplanned 
triangulation study. Hinds and Young (1987) identified the 
specific informational areas associated with each method and then 
planned triangulation strategies in areas where they hypothesized 
that methods tapped the same information. While the authors did 
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not construct a matrix to show the informational cross over among 
and between methods, such a matrix could easily be constructed 
based on their discussion. 

Summary > Seventy-nine percent of the researchers did not 
address the comparability of questions or information associated 
with different methods. Perhaps researchers are taking 
comparability for granted. Perhaps comparability is not an issue 
to researchers. Or perhaps researchers are not sure how to 
demonstrate comparability. Hinds and Young (1987) provide a good 
model for researchers to follow in addressing the comparability 
of questions and information related to different methods. 

5. How do researchers justify the expectation of 
convergence as more valid findings and determine its existence? 

Issues. The theoretical notion of triangulation rests on 
the ideas that irrelevant error associated with individual 
methods can be canceled out and that what is left is more valid 
findings. The emphasis on the potential benefits of 
"triangulating on truth" seems to set the detection of 
convergence as a desirable and dominant goal of mixed method 
triangulation (Mathison, 1988). 

As Mathison (1988) notes, this emphasis on convergence may 
promote the complacent acceptance of convergence as "truth". 
Researchers may "see" what they expect to see . . . which is 
convergence. In this regard^ convergence may be more of a state 
of mind (Mathison, personal communication^ 1990) rather than an 
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empirical entity, and this state of mind might undermine the 
skepticism that seems so crucial to meaningful research. 

Empirically, convergent results may be indicative of 
something other than more valid findings. According to Shetland 
and Mark (1987, p. 91), convergence may actually be 
"pseudoconvergence", an agreement or convergence that does not 
represent accurate or valid data, findings or interpretations. 
Theoretically, pseudoconvergence can occur as a result of several 
factors: (1) a constant source of substantive or methodological 
error operating across methods (Cook, 1985), (2) a constant 
direction of substantive or methodological error operating across 
methods (Cook, 1985) or (3) a set of independent methods 
investigating different questions, processes or variables 
(Shetland & Mark, 1987). Thus, convergence may actually be an 
accumulation of error rather than an accumulation of "truth" . 
It seems reasonable to ask how researchers determine and justify 
that convergence is not something other than "truth" . 

Results and implications. Three themes emerged from the 
data in regard to the ways in which researchers justified the 
expectation of convergence. In two cases (9%) by Erickson (1988) 
and Hinds and Young (1987), researchers justified their 
expectation of convergence as more valid findings based on the 
hypothesis that they had selected sets of methods that possessed 
counteracting biases (i.e., methodological justification). It is 
interesting to note that these researchers did not include 
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simultaneous and independent implementation in their 
justifications for expecting convergence. In most cases (61%)^ 
researchers did not address the expectation of convergence before 
interpreting the results of their studies (i.e., nonexistent 
^stif ication) . In seven cases (30%), researchers seemed to 
imply that convergence was expected based on the expressed but 
not generally explained hypothesis that different methods were 
focused on the same entity of investigation (i.e., substantive 
justification) « 

In nursing science, convergence seemed to be defined as a 
"consistency across findings". Consistency of findings seemed to 
be used as evidence of valid convergence: "Where the results of 
applying different methods reinforce each other, it is not 
unreasonable to assume they are valid" (Pearson, Durant & Punton, 
1989). Validity seemed to be equated with consistency, and 
researchers seemed more concerned with demonstrating consistency 
than with justifying the expectation of convergence as more valid 
results. 

Two themes emerged from the data in regard to the ways in 
which researchers demonstrated consistency across findings. One 
theme seemed more quantitative in nature. It involved the 
quantification of qualitative results and the subsequent analysis 
of the "quantified qualitative data" and the raw quantitative 
data through the use of correlations or percentages. Significant 
positive correlations of as low as .26 were used as evidence of 
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convergence. Researchers did not address the rules of evidence 
they consider in determining which correlations and percentages 
supported convergence and which did not. 

The second theme seemed more cognitive or perhaps more 
qualitative in nature. It involved the demonstration of a 
conceptual congruence between infoirmation obtained from the 
different methods. Researchers seemed to be most concerned with 
comparing substantive themes obtained from qualitative and 
quantitative analyses. An example of this pattern is indicated 
by a quote: "All adolescents verbally validated the direction of 
change in hopefulness indexed by the HSA score" (Hinds, 1989, p. 
446). The determination of convergence seemed most likely to 
occur at the level of meta-analysis, after both qualitative and 
quantitative results had been analyzed and interpreted 
separately. 

Summary . While some researchers supported their expectation 
of convergence, they did not make a case for convergence as more 
valid findings based on the theoretical directives of independent 
and simultaneous implementation. The lacking attention to 
theoretical directives in determining convergence and in ruling 
out pseudoconvergence , and the possible equating of consistency 
and validity lends support to Mathison's (1988) fear that there 
is complacent acceptance of convergence as "truth" in practice. 
However, researchers in nursing science did not refer to 
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convergence as "truth". Rather, they seemed to perceive 
convergence as evidence of substantive issues worthy of 
discussion in informational rather than methodological terms. 

6. What kinds of outcomes do actually result from attempts 
at mixed method triangulation? 

Issyies , In the literature, convergent information is cited 
as the most important outcome of mixed method triangulation. 
Repetitive information, whether it is referred to as convergence 
(Campbell & Fiske, 1959; Cook, 1985), consistency, corroboration 
(Rossman & Wilson, 1985; 1991), or triangulation (Denzin, 1970, 
1978; Webb et al., 1966) is sometimes epitomized as an ultimate 
goal of social science research. Wimsatt in Kaplan (1964, p. 
140) has referred to convergence as a "criterion of reality." 
Similarly^ Cook (1985, p. 48) refers to "points of convergence" as 
potential "sources of light." 

As indicated by Mathison (1988), the emphasis placed on 
obtaining convergence in mixed method triangulation may result in 
the failure to realize or acknowledge other benefits of mixed 
method triangulation (i.e., inconsistency of results that do not 
support a single proposition and contradiction of results that 
provoke new questions or explanations) (p. 15). Furthermore, 
Greene, Caracelli and Graham (1989), have identified five 
purposes or kinds of mixing methods in practice, only one of 
which involves "triangulation", or the use of different methods 
focused on one entity of investigation for the purpose of 
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obtaining similar results. Greene et al.'s (1989) other purposes 
and outcomes of mixing methods include the complementarity and 
expansion of methods and results to gain a more comprehensive 
understanding of an entity, the development of methodology 
through the use of mutually informing methods, and the 
provocation and discovery of new information and questions as a 
result of contradiction. Because there is concern that other 
purposes may be overlooked or considered secondary as a result of 
the emphasis on convergence, it seems reasonable to look at the 
actual outcomes of mixed method triangulation. 

Results and implications. Outcomes in mixed method 
triangulation in nursing science seem to occur at two levels. 
First level outcomes consist of the interpretation of results 
generated from the different methods individually. Second level 
outcomes consist of the interpretation of results from the 
combination of the first level outcomes. For the purposes of 
this discussion, second level outcomes will be referred to as 
meta-outcomes . 

Four different kinds of meta-outcomes emerged from the cases 
of mixed method triangulation in nursing science: repetitive, 
elaborative, contradictory and synergistic. These meta-outcomes 
are consistent with the purposes and outcomes of mixed method 
research identified by Rossman and Wilson (1985, 1991) and 
Greene, Caracelli and Graham (1989). The emergence of different 
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meta-outcoines in mixed method triangulation in nursing science 
may be evidence that researchers are not restricting the role of 
triangulation to the production of convergence. 

In mixed method triangulation in nursing science, repetitive 
meta-outcomes are characterized by a consistency of information 
across different methods* The identification of repetitive meta- 
outcomes in mixed method research in nursing science supports the 
theoretical contention that repetitive information in mixed 
method triangulation practice is sought and obtained. Repetitive 
meta-outcomes were the most frequently cited outcomes of mixed 
method triangulation in nursing science. However, repetitive 
meta-outcomes were not the only outcomes reported by researchers. 
Thus, there may be evidence to refute the hypothesis that 
convergence dominates thinking and practice to the exclusion of 
other meta-outcomes that appear in practice. 

An example of repetitive meta-outcomes in nursing science 
follows. Hinds and Young (1987) investigated the kinds of 
personal wellness goals that patients set and achieved through 
the use of numeric scores on a self -report Likert questionnaire, 
themes generated from interpretive reports of meetings between 
patients and nurses, and content analysis results from verbal 
responses to open ended interview questions. They compared first 
level outcomes from scores, themes and categories and found 
consistency across inferences drawn from the results of different 
methods. The consistency across inferences constitutes a 
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repetitive raeta-outcoiae in Hinds and Young's investigation. 
Repetitive meta-outcoxnes are representative of Rossman and 
Wilson's (1985) corroboration of findings and of Greene et al.'s 
(1989) "... convergence, corroboration, correspondence of results 
.." (p. 259). 

Elaborative meta-outcomes in nursing science are those that 
seem to result in a more comprehensive understanding of an entity 
of investigation. While the intent in triangulation seems to be 
to obtain repetitive meta-outcomes , researchers obtained 
elaborative outcomes when methods revealed different sorts of 
substantive information than that which was expected in the 
original triangulation design. The fact that elaborative meta- 
outcomes emerge in triangulation studies brings into question the 
ability of researchers to predict the substantive targets of 
different methods. 

An example of elaborative meta-outcomes follows: Giles, 
Bradbard and Endsley (1987) used triangulation strategies to 
investigate the impact of child life interns on children and 
adults in a community hospital. Time sampling and participant 
observation were used to answer the same research questions: (1) 
Where are the children? (2) What are the children doing? (3) 
Who is with the children? (4) What are the parents and children 
doing? The two methods did provide repetitive or convergent 
information regarding the research questions. 
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However^ as the investigation progressed^ it became evident 
that in addition to providing the same information^ each method 
provided new information about the entity of investigation that 
the other method did not provide. While both methods provided 
information about "what" was going on, time sampling revealed 
"how often" things were occurring and participant observation 
provided inforonation about "how, why and in what context" things 
were happening. Although repetitive information was sought 
through the use of mixed method triangulation, elaborative meta- 
outcomes were also obtained. In elaborative meta-outcomes, 
differences in methods seemed to produce different but more 
comprehensive and complementary information. Elaborative meta- 
outcomes seem representative of Rossman and Wilson's (1985) 
elaboration of findings and the desired outcomes of Greene et 
al.'s (1989) complementarity and expansion purposes. 

Contradictory meta-outcomes seemed to occur when information 
generated by different methods was incongruent. Contradictory 
meta-outcomes may be perceived as the opposite of repetitive 
outcomes in the sense that a comparison of the information 
obtained from different methods focused on the same entity 
reveals oppositional findings where similar findings were 
expected. Contradictory meta-outcomes seem representative of the 
concept of nonconvergence (Campbell & Fiske, 1959; Cook, 1985? 
Mark & Shetland, 1987). 
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An example of a contradictory raeta-outcome follows. Glik 
(1986) investigated health care behavior of Rwandese mothers. 
The results of a deductive, structured survey indicated that 42% 
of the mothers took their children to a health care clinic when 
the children had fevers and that 38% gave their children oral 
rehydration salts (ORS) when the children had diarrhea. In a 
focus group, all (100%) mothers reported taking children with 
fevers to a health clinic and all (100%) mothers said they had 
used ORS and found it to be beneficial. Percentages from the 
survey and percentages from the focus group constitute first 
level outcomes. The discrepancies in the percentages across 
methods are examples of nonconvergence and constitute 
contradictory meta-outcomes. 

The last type of meta-outcome observed in mixed method 
triangulation in nursing science is a synergistic meta-outcome. 
A synergistic meta-outcome is marked by its newness. It is a 
'whole that is greater than the sum of its parts'. It is what 
Greene et al. (1989) and Rossman & Wilson (1985) refer to as 
"initiation". An example follows. 

In a study of bereavement related to the Mount Saint Helens 
eruption (Murphy, 1988), a combination of first level outcomes 
from structured interviews and standardized instruments led to 
the reconceptualization of the process of "recovery" in relation 
to victims of the volcanic eruption. Instead of returning to 
conditions as they were before, it became evident that people's 
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lives were "forever changed." The process of recovery was 
reconceptualized as a process of adaptation where in people 
reported progress in terras of adjustment rather than recovery in 
terms of returning to past conditions. 

With synergistic meta-outcomes ^ different methods produce 
new and different information- In nursing science, these 
included (a) new questions, (b) new solutions and (c) new 
understandings. In several cases, synergistic meta-outcomes 
allowed researchers to see errors in previous conceptualizations 
or instrumentation. Synergistic meta-outcomes seemed to be the 
result of contradictory as well as elaborative meta-outcomes . 

Summary . While repetitive meta-outcomes seem to be the 
intended outcomes of mixed method triangulation in nursing 
science, researchers did obtain other meaningful meta-outcomes: 
elaboration, contraciiction and synergy. Based on results from 
nursing science, triangulation can h*'i perceived as a vehicle for 
more than the production of convergent findings. Future research 
may involve a comparison of the value of convergent, elaborative, 
contradictory and synergistic meta-outcomes from the perspective 
of mixed method researchers. It may be that synergistic 
reconceptualizations rather than convergence are the most valued 
meta-outcomes of mixed method triangulation. 
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7. How is nonconvergence resolved? 

Tssues, While convergence seems to be the desired result 
of mixed method triangulation, nonconvergence may occur. 
Shetland and Mark (1987, p. 91) discuss several theoretical 
explanations for nonconvergence: (a) methods that avidress 
different parts of an entity of investigation, (b) methods that 
measure different parts of an entity of investigation, (c) 
methods that tap different intensities of an entity of 
investigation, and (d) methods that have different degrees of 
error that is not counteracted in the research process . Cook 
(1985) notes that nonconvergence presents a researcher with a 
puzzle to be solved. It may be that when ir^dependent methods 
have been employed and focused on the same entity and when 
convergence is expected that nonconvergence may be most likely to 
yield very significant substantive meaning. However, an emphasis 
on methodological explanations for nonconvergence may inhibit the 
development of substantive explanations. 

Results and implications. Researchers did acknowledge 
nonconvergent results and demonstrated different approaches to 
dealing with the contradictory meta-outcomes . One researcher 
reported the contradictory results but did not discuss any 
potential resolutions. Some researchers offerad methodological 
reasons to explain the contradiction. For example, in the case 
where Glik et al. (1986) obtained discrepancies in percentages 
across methods, they resolved the discrepancy by attributing the 



ERLC 



39 



Mixed Method Triangulation 

Page 39 

higher percentages obtained from the focus group interviews to 
the tendency of participants to want to please researchers by 
giving what they perceived as the correct or desired responses. 
Methodological explanations for nonconvergence given by other 
researchers included: lacking sensitivity or inappropriateness 
of scales, insufficient sample size for statistical significance, 
reactivity bia« and testing effect, poor design, high mortality 
rate and methods focused on different entities. 

Some researchers offered substantive reasons for 
nonconvergence. For example, Aroian and Patsdaughter (1989) 
combined several different aspects and dimensions of 
psychological distress in Polish immigrants: participants' self 
ratings on the Brief Symptom Inventory (BSI), participants' 
verbal self --reports of the migration and resettlement experience, 
participant observation of social interactions, and clinical 
assessment of mental status. Individual scores and group means 
on the psychoticism subscale of the BSI inventory, were high. 
The scores were indicative of withdrawal, isolation, thought 
disorder, and schizoid life-style. 

Observations and verbal reports seemed to support the 
psychoticism scores. For instance, observations of interaction 
indicated a high degree of interpersonal distance and verbal 
reports of social relationships indicated that people questioned 
the authenticity of social relationships. However, clinical 
assessments did not indicate signs of thought disorder or 
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schizophrenia. Through interviews it was determined that the 
Polish participants did not trust or socialize much because of a 
cultural response to Communist control rather than because they 
were psychotic as indicated on the BSI. The incongruence of 
inforraatior was resolved substantively. 

For others seeking substantive resolutions, nonconvergence 
acted as a springboard to new conceptualizations or synergistic 
meta-outcomes . And others approached the task of resolving 
nonconvergence by developing a series of plausible rival 
hypotheses and then constructing arguments to support or refute 
each hypothesis. Examples of this latter strategy are evident in 
studies by Hinds and Young (1987), Erickson (1988) and Rowatt 
(1983) . 

Summary > Nonconvergent findings wore cited 10 times in 
mixed method triangulation studies in nursing science. Thus, 
there is evidence that researchers are not necessarily 
overlooking the divergent findings that they obtain. In 
addition, researchers seemed to reinforce Cook^F (1985) idea that 
divergent results offer an opportunity for problem solving. Both 
methodological and substantive explanations were given to resolve 
the contradictory results. Substantive explanations were offered 
more frequently than methodological explanations, thereby 
contradicting the notion that methodological explanations inhibit 
the development of substantive explanations. 



41 



Mixed Method Triangulation 

Page 41 

Theory and Practice Compared; Summary 

Results of this investigation indicate that researchers in 
nursing science did not necessarily apply the theoretical 
direcMves of the heterogeneity of irrelevancy assumption to 
their practice of mixed method triangulation. Researchers did 
not discuss the heterogeneity of irrelevancy assumption in their 
work. As a group, nursing science researchers did not employ 
independent or simultaneous implement cion of different methods. 
They did not usually construct technical or methodological 
justifications of convergence as more valid findings. Instead, 
researchers seemed to focus their attention on substantive or 
informational issues rather than on technical or methodological 
issues . 

The lack of attention given to theoretical implications of 
the heterogeneity of irrelevancy assumption by nursing science 
researchers may be explained in sever© different ways. First, 
researchers may not be aware of the theoretical implications ot 
the heterogeneity of irrelevancy assumption for mixed method 
triangulation. Second, researchers may be aware of the 
theoretical implications and consider a counteraction of bias to 
be robust to violations. Third, researchers may find that it is 
not feasible to actualize the theoretical implications in 
practice. Fourth, researchers may feel that the conventional 
theory based on the heterogeneity of irrelevancy assumption has 
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little to do with the conception of triangulation that they 
actualize in practice. These four hypotheses may provide the 
foci for future investigations. 

Results of this investigation suggest that researchers in 
nursing science are practicing a conception of triangulation that 
is driven by substantive rather than methodological/technical 
concerns. Nursing science researchers tended to construct 
substantively-based justifications for designs and for 
interpretations of results. This is in contrast to the 
formulation of more technical features on which the conventional 
conception of triangulation seems to rest. Yet, this situation 
does seem consistent with Barone's (1990) idea that when research 
is guided by substantive considerations, methodological issues 
fall into place. 

These results suggest that the heterogeneity of irrelevancy 
assumption may not be a useful guideline for some practitioners 
of mixed method triangulation in nursing science. Examples from 
the practice of mixed method triangulation in nursing science may 
provide data-based guidelines for researchers who are trying to 
(a) construct rationales for method selection, (b) resolve 
divergent results and (c) realize multiple meta-outcomes from 
triangulation. More research is needed to develop an alternative 
conception of mixed method triangulation that is grounded in the 
perspectives of researchers conducting mixed method 
triangulation . 
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Appendix A: Mixed Method Purposes (Greene et al., 1989) 

1. "TRIANGULATION seeks convergence, corroboration, 

correspondence of results frora the different methods" 
(Greene et al., 1989, p. 259). 

2. "COMPLEMENTARITY seeks elaboration, enhancement, 

illustration, clarification of the results from one method 
with the results from the other method" (Greene et al., 
1989, p. 259). 

3. "DEVELOPMENT seeks to use the results from one method to help 

develop or inform the other method, where development is 
broadly construed to include sampling and implementation, as 
well as measurement decisions" (Greene et al., 1989, p. 
259) . 

4. "INITIATION seeks the discovery of paradox and contradiction, 

new perspectives of frameworks, the recasting of questions 
or results from one method with questions or results from 
the other method" (Greene et al., 1989, p. 259). 

5. "EXPANSION seeks to extend the breadth and range of inquiry 

by using different methods for different inquiry components" 
(Greene et al., 1989, p. 259). 
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